Bayesian Motion Estimation and Segmentation Bayesian Motion Estimation and Segmentation

Estimating motion in scenes containing multiple moving objects remains a diicult problem in computer vision yet is solved eeortlessly by humans. In this thesis we present a computational investigation of this astonishing performance in human vision. The method we use throughout is to formulate a small number of assumptions and see the extent to which the optimal interpretation given these assumptions corresponds to the human percept. For scenes containing a single motion we show that a wide range of previously published results are predicted by a Bayesian model that nds the most probable velocity eld assuming that (1) images may be noisy and (2) velocity elds are likely to be slow and smooth. The predictions agree qualitatively, and are often in remarkable agreement quantitatively. For scenes containing multiple motions we introduce the notion of \smoothness in layers". The scene is assumed to be composed of a small number of surfaces or layers, and the motion of each layer is assumed to be slow and smooth. We again formalize these assumptions in a Bayesian framework and use the statistical technique of mixture estimation to nd the predicted percept. Again, we nd a surprisingly wide range of previously published results that are predicted with these simple assumptions. We discuss the shortcomings of these assumptions and show how additional assumptions can be incorporated into the same framework. Taken together, the rst two parts of the thesis suggest that a seemingly complex set of illusions in human motion perception may arise from a single computational strategy that is optimal under reasonable assumptions. The third part of the thesis presents a computer vision algorithm that is based on the same assumptions. We compare the approach to recent developments in motion segmentation and illustrate its performance on real and synthetic image sequences. Acknowledgments I thank my advisor, Ted Adelson, for his wisdom, support and encouragement. I hope that in my own career I will be able to follow the example that he sets | his endless curiosity about how the brain works, his brilliant insights into complicated phenomena and the fundamental honesty with which he conducts himself as a scientist. It has also been a pleasure to interact with Ted as a colleague and I will fondly remember our conversations on topics ranging from the theory of evolution to root beer. Thanks also to the members of my thesis committee | Mike Jordan, Tommy Poggio and …